TALKING HEADS: Physical, Linguistic and Cognitive Issues in Facial Animation

Facial modeling and animation are increasingly receiving attention in the graphics and artificial intelligence (AI) research communities, both of which share the common goal of synthesizing believable, simulated agents. While computer graphics researchers have been primarily concerned with physical and anatomical aspects of facial movements, AI researchers and cognitive scientists have focused on understanding and modeling the motivation behind those movements and expressions. The combination of these two avenues of research may eventually lead to agents that can interact autonomously, with humans or with each other, bearing faces that believably model the underlying meaning of the interactions. While such synthetic speaking faces are undoubtedly useful for cognitive research, their practical applications are also vast in number, encompassing such diverse fields as medicine, education, telecommunications and the entertainment industry. Facial expressions have fascinated mankind for centuries, depicting our personalities, emotions, moods, and speech patterns. When interacting with one another, we examine faces to gather information that may not be conveyed by the spoken words alone. We are able to recognize faces from our earliest days of childhood, making them perhaps our most basic communicative tool. In fact, it is precisely this familiarity with facial expressions that makes it difficult for researchers to simulate them, for we are always acutely aware of the smallest discrepancies between real and synthetic faces. The purpose of this course is to give on overview of the current state of the art in the field of facial modeling and animation, paying particular attention to the relevant linguistic and cognitive issues in determining facial expressions. In Part I of the course, we lay the groundwork for discussing facial animation by presenting an introduction to facial anatomy and the physical aspects of facial modeling. In Part II, we describe the role of intonational patterns in speech, which partially determine the facial movements that accurately reflect the semantic content of the speech. Finally, in Part III, we describe the emotional and linguistic aspects of facial expression and describe an implemented system for automatically generating speech with context-appropriate intonation and corresponding facial animation.